Death and Suicide in Universal Artificial Intelligence

Reinforcement learning (RL) is a general paradigm for studying intelligent behaviour, with applications ranging from artificial intelligence to psychology and economics. AIXI is a universal solution to the RL problem; it can learn any computable environment. A technical subtlety of AIXI is that it is defined using a mixture over semimeasures that need not sum to 1, rather than over proper probability measures. In this work we argue that the shortfall of a semimeasure can naturally be interpreted as the agent's estimate of the probability of its death. We formally define death for generally intelligent agents like AIXI, and prove a number of related theorems about their behaviour. Notable discoveries include that agent behaviour can change radically under positive linear transformations of the reward signal (from suicidal to dogmatically self-preserving), and that the agent's posterior belief that it will survive increases over time.Comment: Conference: Artificial General Intelligence (AGI) 2016 13 pages, 2 figure

Count-Based Exploration in Feature Space for Reinforcement Learning

We introduce a new count-based optimistic exploration algorithm for Reinforcement Learning (RL) that is feasible in environments with high-dimensional state-action spaces. The success of RL algorithms in these domains depends crucially on generalisation from limited training experience. Function approximation techniques enable RL agents to generalise in order to estimate the value of unvisited states, but at present few methods enable generalisation regarding uncertainty. This has prevented the combination of scalable RL algorithms with efficient exploration strategies that drive the agent to reduce its uncertainty. We present a new method for computing a generalised state visit-count, which allows the agent to estimate the uncertainty associated with any state. Our \phi-pseudocount achieves generalisation by exploiting same feature representation of the state space that is used for value function approximation. States that have less frequently observed features are deemed more uncertain. The \phi-Exploration-Bonus algorithm rewards the agent for exploring in feature space rather than in the untransformed state space. The method is simpler and less computationally expensive than some previous proposals, and achieves near state-of-the-art results on high-dimensional RL benchmarks.Comment: Conference: Twenty-sixth International Joint Conference on Artificial Intelligence (IJCAI-17), 8 pages, 1 figur

A search for ultra-high-energy photons at the Pierre Auger Observatory exploiting air-shower universality

The Pierre Auger Observatory is the most sensitive detector to primary photons with energies above ∼0.2 EeV. It measures extensive air showers using a hybrid technique that combines a fluorescence detector (FD) with a ground array of particle detectors (SD). The signatures of a photon-induced air shower are a larger atmospheric depth at the shower maximum (X$_{max}$) and a steeper lateral distribution function, along with a lower number of muons with respect to the bulk of hadron-induced background. Using observables measured by the FD and SD, three photon searches in different energy bands are performed. In particular, between threshold energies of 1-10 EeV, a new analysis technique has been developed by combining the FD-based measurement of X$_{max}$ with the SD signal through a parameter related to its muon content, derived from the universality of the air showers. This technique has led to a better photon/hadron separation and, consequently, to a higher search sensitivity, resulting in a tighter upper limit than before. The outcome of this new analysis is presented here, along with previous results in the energy ranges below 1 EeV and above 10 EeV. From the data collected by the Pierre Auger Observatory in about 15 years of operation, the most stringent constraints on the fraction of photons in the cosmic flux are set over almost three decades in energy

Study on multi-ELVES in the Pierre Auger Observatory

Since 2013, the four sites of the Fluorescence Detector (FD) of the Pierre Auger Observatory record ELVES with a dedicated trigger. These UV light emissions are correlated to distant lightning strikes. The length of recorded traces has been increased from 100 μs (2013), to 300 μs (2014-16), to 900 μs (2017-present), to progressively extend the observation of the light emission towards the vertical of the causative lightning and beyond. A large fraction of the observed events shows double ELVES within the time window, and, in some cases, even more complex structures are observed. The nature of the multi-ELVES is not completely understood but may be related to the different types of lightning in which they are originated. For example, it is known that Narrow Bipolar Events can produce double ELVES, and Energetic In-cloud Pulses, occurring between the main negative and upper positive charge layer of clouds, can induce double and even quadruple ELVES in the ionosphere. This report shows the seasonal and daily dependence of the time gap, amplitude ratio, and correlation between the pulse widths of the peaks in a sample of 1000+ multi-ELVES events recorded during the period 2014-20. The events have been compared with data from other satellite and ground-based sensing devices to study the correlation of their properties with lightning observables such as altitude and polarity